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Situation  assessment  (SA)  involves  deriving  relations  among  en¬ 
tities,  e.g.,  the  aggregation  of  object  states  (i.e.,  classification  and 
location).  While  SA  has  been  recognized  in  the  information  fusion 
and  human  factors  literature,  there  still  exist  open  questions  regard¬ 
ing  knowledge  representation  and  reasoning  methods  to  afford  SA. 
For  instance,  while  lots  of  data  is  collected  over  a  region  of  interest, 
how  does  this  information  get  presented  to  an  attention  constrained 
user?  The  information  overload  can  deteriorate  cognitive  reason¬ 
ing  so  a  pragmatic  solution  to  knowledge  representation  is  needed 
for  effective  and  efficient  situation  understanding.  In  this  paper, 
we  present  issues  associated  with  Level  2  Information  Fusion  (Sit¬ 
uation  Assessment)  including:  (1)  user  perception  and  perceptual 
reasoning  representation,  (2)  knowledge  discovery  process  models, 
(3)  procedural  versus  logical  reasoning  about  relationships,  (4)  user- 
fusion  interaction  through  performance  metrics,  and  (5)  syntactic 
and  semantic  representations.  While  a  definitive  conclusion  is  not 
the  aim  of  the  paper,  many  critical  issues  are  proposed  in  order  to 
characterize  future  successful  strategies  for  knowledge  representa¬ 
tion,  presentation,  and  reasoning  for  situation  assessment. 
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Situation  assessment  (SA)  is  an  important  part  of 
the  information  fusion  (IF)  process  because  it  (1)  is  the 
purpose  for  the  use  of  IF  to  synthesize  the  multitude 
of  information,  (2)  provides  an  interface  between  the 
user  and  the  automation,  and  (3)  focuses  data  collection 
and  management.  Hall  and  Llinas  (Table  I)  have  listed 
a  variety  of  techniques  that  need  to  be  solved  for  SA 
to  be  viably  implemented  in  real  systems  [15].  Since 
the  late  1990s  there  has  been  few  cumulative  updates  in 
the  progress  of  SA  and  still  there  are  remaining  issues 
and  challenges.  During  the  FUSION05  conference,  Ivan 
Kadar  organized,  moderated,  and  participated  in  a  panel 
discussion  with  invited  leading  experts  to  elicit  and 
summarize  current  issues  and  challenges  in  SA  that  need 
to  be  researched  in  the  next  decade. 

1.1.  Panel  Participants,  Topics,  and  Perspectives 

This  paper  serves  as  a  retrospective  view  of  the 
panel  discussion  that  was  held  in  July  2005.  In  this 
format,  we  list  our  retrospective  and  annotated  view  of 
the  panel  information  in  a  condensed  (bulletized)  format 
to  make  it  easier  for  the  reader  to  assimilate  the  general 
concepts.  Due  to  space  limitation,  only  a  few  key  issues 
are  expanded  on  in  text  format. 

•  Organizer:  Ivan  Kadar,  Interlink  Systems  Sciences, 
Inc. 

•  Co-Organizers:  Subrata  Das,  Charles  River  Analyt¬ 
ics  and  Mieczyslaw  M.  Kokar,  Northeastern  Univer¬ 
sity 

•  Moderators:  Ivan  Kadar,  Interlink  Systems  Sciences, 
Inc.  and  James  Llinas,  SUNY  at  Buffalo 

•  July  26,  2005  FUSION  2005— The  8th  Interna¬ 
tional  Conference  on  Information  Fusion,  July  25-28, 
Philadelphia,  PA 

PARTICIPANTS  AND  PRESENTATION  TITLES 

•  “Knowledge  Representation  Issues  in  Perceptual  Rea¬ 
soning  Managed  Situation  Assessment”  Ivan  Kadar, 
Interlink  Systems  Sciences,  Inc.,  Lake  Success,  NY 

•  “Knowledge  Representation  Requirements  for  Situ¬ 
ation  Awareness”  John  Salerno,  Douglas  Boulware, 
Raymond  Cardillo,  Air  Force  Research  Laboratory, 
Rome  Research  Site,  NY 

•  “Situation  Assessment:  Procedural  versus  Logical” 
Mieczyslaw  M.  Kokar,  Department  of  Elect.  &  Com¬ 
puter  Eng.,  Northeastern  University,  Boston,  MA 

•  “Tactical  Situation  Assessment  Challenges  and  Impli¬ 
cations  for  Computational  Support”  Gerald  M.  Pow¬ 
ell,  U.S.  Army  RDECOM  CERDEC  I2WD,  Ft.  Mon¬ 
mouth,  NJ 

•  “Situation  Assessment  in  Urban  Combat  Environ¬ 
ments”  Subrata  Das,  Charles  River  Analytics,  Inc., 
Cambridge,  MA 
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TABLE  I 

SA  Challenges  and  Limitations — Hall  and  Llinas,  [15] 


JDL  Process  Processing  Description 

Current  Status 

Challenges  and  Limitations 

Level  2  Develops  a  description  of  current 

relationships  among  objects  and 
events  in  the  context  of  the 
environment  (i.e.,  situation 
assessment) 

Numerous  prototypes 

Dominance  by  Knowledge-Based 
Systems  (KBS) 

— Blackboard  methods 
— Rule-based  representation 
— Logical  templates 

KBS  experiments 

— Case  based  reasoning,  Fuzzy  Logic 
Non-real  time  implementation 

Dominated  by  prototypes 

No  experience  on  scaling  to  field 
models 

“Excedrin”  cognitive  models 

Difficult  KB  development 

Perfunctory  Test  &  Evaluation 

Integration  of  identity/kinematic  data 

•  “Representation  and  Contribution-Integration  Chal¬ 
lenges  in  Collaborative  Situation  Assessment”  Daniel 
D.  Corkill,  University  of  Massachusetts,  Amherst, 
MA 

•  “Human-Aided  Multi-Sensor  Fusion”  Enrique  H. 
Ruspini,  et  al.,  Artificial  Intelligence  Center,  SRI  In¬ 
ternational,  Menlo  Mark,  CA 

•  “DFIG  Level  5  (User  Refinement)  issues  supporting 
Level  2  (Situation  Assessment)”  Erik  Blasch,  AFRL, 
WPAFB,  OH 

1.2.  Common  Themes 

While  discussion  of  individual  research  results  by 
the  participants  highlighted  specific  key  issues,  there 
were  common  themes  that  resulted  from  the  panel  dis¬ 
cussion.  The  common  themes  were: 

COMMON  ISSUES 

•  User — The  SA  process  includes  perceptual,  interac¬ 
tive,  and  human  control 

•  Process  models — updating  behavioral  models  (e.g. 
Bayes  Nets,  procedural/logical,  perceptual,  learning) 

•  Context — operational  situation  (i.e.,  dependent  on  the 
current  state  of  the  environment) 

•  Meaning — semantics  and  syntax  issues  (formal  meth¬ 
ods,  ontologies) 

•  Metrics — develop  a  standard  set  of  metrics  (e.g.  trust, 
bounds,  uncertainty) 

COMMON  CHALLENGES 

•  Explanation  of  process — evidence  accumulation  and 
contradiction  in  knowledge  representation  and  rea¬ 
soning 

•  Graphical  displays  to  facilitate  inferential  chains,  col¬ 
laborative  interaction,  and  knowledge  presentation 

•  Interactive  control  for  corrections  and  utility  assess¬ 
ment  for  knowledge  management 

2.  SITUATION  AWARENESS/SITUATION 
ASSESSMENT 

There  are  two  main  communities  that  are  look¬ 
ing  at  situational  information  (i.e.,  Situation  Awareness 


Fig.  1.  Endsley’s  situation  awareness  model. 


(SAW)  and  Situation  Assessment  (SA)):  the  human  fac¬ 
tors  community  and  the  engineering  information  fusion 
(IF)  research  community.  SAW  is  a  mental  state  while 
SA  supports  (e.g.  fusion  products)  that  state.  The  human 
factors  notion  of  SAW  is  being  lead  by  Mica  Endsley 
[12].  For  the  IF  society,  there  are  many  leading  people 
proposing  different  aspects  of  SAW  research.  Research 
is  a  way  to  categorize  developments,  but  another  way  is 
by  applications.  There  are  many  application  communi¬ 
ties  looking  at  SAW  research  including:  military,  med¬ 
ical,  aviation,  security,  and  environmental.  Each  might 
have  differences,  but  the  commonality  rests  in  the  fact 
that  a  multitude  of  data  needs  to  be  synthesized  into  a 
single  operating  picture  (dimensionality  reduction)  [37] . 
Likewise,  the  salient  information  needs  to  be  provided 
to  the  user  to  assist  the  user  in  completing  their  mission 
tasks. 

2.1.  Situational  Awareness  Models 

The  Human  in  the  Loop  (HIL)  of  a  semi-automated 
system  must  be  given  adequate  situation  awareness. 
According  to  Endsley  “SAW  is  the  perception  of  the 
elements  in  the  environment  within  a  volume  of  time 
and  space,  the  comprehension  of  their  meaning,  and  the 
projection  of  their  status  in  the  near  future.”  [12].  This 
now-classic  model,  shown  in  Fig.  1,  translates  into  3 
levels: 

•  Level  1  SAW — Perception  of  elements  in  the  envi¬ 
ronment 
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Fig.  2.  Fusion  situation  awareness  model  [4], 


•  Level  2  SAW — Comprehension  of  the  current  situa¬ 
tion 

•  Level  3  SAW — Projection  of  future  states 

Operators  of  dynamic  systems  use  their  SAW  in  de¬ 
termining  their  actions.  To  optimize  decision  making, 
the  SAW  provided  by  an  IF  system  should  be  as  precise 
as  possible  as  to  the  objects  in  the  environment  (Level  1 
SAW).  A  SA  approach  should  present  a  fused  represen¬ 
tation  of  the  data  (Level  2  SAW)  and  provide  support  for 
the  operator’s  projection  needs  (Level  3  SAW)  in  order 
to  facilitate  the  operator’s  goals.  From  the  SA  model 
presented  in  Fig.  1,  workload  is  a  key  component  of  the 
model  that  affects  not  only  SAW,  but  also  the  decision 
and  reaction  time  of  the  user. 

2.2.  User  Fusion  Model 

As  another  example,  the  Situational  Model  compo¬ 
nents  [32],  shown  in  Fig.  2,  developed  by  Roy,  show 
the  various  information  needs  to  provide  the  user  with 
an  appropriate  SAW.  To  develop  the  SA  model  further, 
we  note  that  the  user  must  be  primed  for  situations  to 
be  able  to  operate  faster,  and  more  effectively. 

A  fusion  system  must  satisfy  the  user’s  functional 
needs  and  extend  their  sensory  capabilities.  Of  inter¬ 
est  to  the  information  fusion  community  are  IF  sys¬ 
tems  which  translate  data  about  a  region  of  interest  into 
knowledge,  or  at  least  information  over  which  the  hu¬ 
man  can  reason  and  make  decisions.  A  user  fuses  data 
and  information  over  time  and  space  and  acts  through 
their  world  mental  model — whether  it  be  in  the  head 
or  with  graphical  displays,  tools,  and  techniques.  The 
current  paradigm  for  fusion  research,  shown  in  Fig.  3, 
is  called  the  user-fusion  model  [5]. 


Fig.  3.  User  fusion  model. 


2.3.  Perceptual  Reasoning  Managed  Situation 
Assessment 

“Knowledge  Representation  Issues  in  Perceptual 
Reasoning  Managed  Situation  Assessment”  Ivan  Kadar 
The  IF  community  has  had  several  definitions  of  SA 
over  time.  The  JDL  Model  [14],  defined  SA  as  “estima¬ 
tion  and  prediction  of  relations  among  entities,  to  in¬ 
clude  force  structure  and  cross  force  relations,  commu¬ 
nications  and  perceptual  influences,  physical  context, 
etc.”  DSTO  [11,  22]  defined  SA  as  “an  iterative  process 
of  fusing  the  spatial  and  temporal  relationships  between 
entities  to  group  them  together  and  form  an  abstracted 
interpretation  of  the  patterns  in  the  order  of  battle  data.” 
Issues  with  the  SA  definitions,  and  some  subsequent 
models  based  on  these  definitions  are: 

•  not  domain  independent, 

•  do  not  incorporate  human  thought  processes,  human 
perceptual  reasoning,  the  ability  to  control  sensing 
and  essence  of  response  time, 
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•  imply  use  of  limited  a  priori  information, 

•  and  only  imply  potential  for  new  knowledge  cap¬ 
ture. 

Therefore,  the  desired  properties  of  SA  are: 

•  One  needs  the  ability  to  control  Levels  1—4  of  Data 
Fusion  processes  for  knowledge  capture  in  SA 

•  SA  is  to  establish  relationships  (not  necessarily  hi¬ 
erarchical)  and  associations  among  entities,  it  should 
anticipate  with  a  priori  knowledge  in  order  to  rapidly 
gather,  assess,  interpret  and  predict  what  these  rela¬ 
tionships  might  be;  it  should  plan,  predict,  anticipate 
again  with  updated  knowledge,  adaptively  learn,  and 
control  the  fusion  processes  for  optimum  knowledge 
capture  and  decision  making 

•  These  features  are  similar  to  the  characteristics  of 
human  perceptual  reasoning 

•  Therefore  it  is  conjectured  that  the  “optimum”  SA 
system  should  emulate  human  thinking  as  much  as 
possible 

As  a  matter  of  fact,  the  godfather  of  the  Internet 
and  knowledge  representation,  Vannevar  Bush  [8]  in 
his  famous  1945  essay,  “As  We  May  Think”  stated, 
op.  cit.,  “The  human  mind  does  not  work  that  way 
hierarchically.  It  operates  by  association.”  Spatial  and 
temporal  associations  are  key  ingredients  of  Perceptual 
Reasoning  Model  (PRM). 

The  goal  is  the  perceptual  reasoning  model  which  is 
viewed  as  a  “meta-level  information  management  sys¬ 
tem,”  as  shown  in  Fig.  4.  PRM  consists  of  a  feed¬ 
back  planning/resource  control  system  whose  interact¬ 
ing  elements  are:  “assess,”  “anticipate”  and  “predict” 
[16-18], 

•  Gather/Assess  current.  Anticipate  future  (hypothe¬ 
ses),  and  Predict  information  requirements  and  mon¬ 
itor  intent, 

•  Plan  the  allocation  of  information/sensor/system  re¬ 
sources  and  acquisition  of  data  through  the  control 
of  a  separate  distributed  multisource  sensors/systems 
resource  manager  (SRM), 

•  Interpret  and  act  on  acquired  (sensor,  spatial  and 
contextual)  data  in  light  of  the  overall  situa¬ 
tion  by  interpreting  conflicting/misleading  informa¬ 
tion. 


Representative  elements  and  knowledge  bases,  as¬ 
sociated  with  the  assess,  anticipate  and  predict  PRM 
modules,  are  categorizable  into:  (1)  functions,  with 
each  function  further  categorized  into  (a)  knowledge  re¬ 
quired,  (b)  knowledge  acquisition  methods,  (c)  knowl¬ 
edge  representation  approaches,  and  (d)  implementa¬ 
tion  techniques.  Specific  knowledge  representation  and 
reasoning  (KRR)  methods  were  discussed  at  the  panel 
highlighting  implementation  issues  and  research  chal¬ 
lenges. 


Issues  for  SA 

1 .  Knowledge — a  priori  and  current 

2.  PROCESS — anticipate  and  gather  facts 

3.  User  queries  instantiation 

4.  Fusion  System  presents  Beliefs 

5.  Need  a  process  model  interface 

KRR  Challenges  for  SA 

1.  Adequacy  of  KRR  (logic,  ontology,  algorithmic, 
probabilistic),  how  to  quantify/measure? 

2.  Expressiveness  of  models  versus  tractability  of  inference 

3.  Managing  Complexity  (how  to  bound  problem  w/incomplete 
knowledge ) 

4.  Data  Information  (How  to  manage  heterogeneous  and 
uncertain  KSs  and  detect  duplicate  or  incomplete  concepts) 

5.  Presentation  of  knowledge  to  different  users  (what  is 
pragmatic?) 


2.4.  Syntactic  Algorithms  and  Semantic  Synonyms 

“Knowledge  Representation  Requirements  for  Situ¬ 
ation  Awareness”  John  Salerno,  Doug  Boulware,  Ray 
Cardillo 

Full  Spectrum  Dominance  (FSD),  as  defined  by 
Joint  Vision  2020,  is  the  ability  to  be  persuasive  in 
peace,  decisive  in  war  and  preeminent  in  any  form  of 
conflict.  FSD  cannot  be  accomplished  without  the  ca¬ 
pability  to  know  what  the  adversary  is  currently  doing 
as  well  as  the  capacity  to  correctly  anticipate  the  ad¬ 
versary’s  future  actions.  This  ability  of  projection  is  an 
element  of  Situation  Awareness  [12,  13].  SA  has  re¬ 
ceived  increased  attention  due  to  its  diverse  applications 
in  a  number  of  problem  domains  including:  asymmet¬ 
ric  threat,  tactical,  cyber,  and  homeland  security  [14], 
Salerno,  et  al.  proposes  an  architecture  that  combines 
the  Endsley  and  JDL  models  (shown  in  Fig.  5)  and  has 
applied  this  model  to  various  strategic,  cyber  and  tactical 
applications  [35]. 

Through  a  display,  a  user  can  (1)  build  a  model 
by  either  editing  an  existing  template/model  or  create 
a  new  one;  (2)  activate/de-activate  existing  models;  or 
(3)  view  active  models  and  any  evidence  that  has  been 
associated  with  the  model  over  time.  Different  political, 
military,  economic,  social,  infrastructure,  and  informa¬ 
tion  models  can  be  accessed  and  the  result  published  (or 
subscribed)  to. 
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Fig.  5.  SA  framework. 


Issues  encountered  in  its  development  mainly  per¬ 
tain  to  evidence  access,  storage,  usage  and  providing 
a  priori  knowledge.  In  order  to  resolve  any  seman¬ 
tic  issues  in  context  and  value,  we  need  to  normalize 
the  data  before  we  can  use  it.  Data  normalization  in¬ 
volves  converting  different  formats  of  the  same  data 
into  a  common  representation.  Dealing  with  semantic 
inconsistencies  is  much  more  difficult.  In  these  cases, 
we  need  to  resolve  synonyms  both  in  what  is  repre¬ 
sented  and  what  the  value  itself  represents.  Two  differ¬ 
ent  labels  can  have  the  same  meaning,  or  two  aliases 
can  represent  the  same  entity.  Finally,  what  level  of 
a  priori  data  is  needed  depends  on  the  context  of  op¬ 
eration. 


Issues  for  SA 

1 .  Lots  of  data  for  analyst,  but  not  able  to  get  it 

2.  Analyst — under  stress  and  fatigue 

3.  What  to  publish  and  subscribe 

4.  Security  issues  in  data  gathering 

Challenges  for  SA 

1.  Syntactic  algorithms  (normalization/transformation) 

2.  Semantic  synonyms  (different  meaning  between  ideas) 

3.  Learning  from  what  is  presented 

4.  People  can  think  of  new  situations 


2.5.  Procedural  versus  Logical 

“Situation  Assessment:  Procedural  versus  Logical” 
Mieczyslaw  M.  Kokar 

Various  terms  have  been  used  to  refer  to  Level  2  fu¬ 
sion  processing:  situation  refinement,  situation  aware¬ 
ness,  situation  development,  relation  estimation  and 
other.  All  of  these  terms  have  a  common  part  in  their 
definition,  i.e.,  all  of  them  require  that  the  definition 
should  include  the  knowledge  of  all  the  relevant  objects 
and  their  kinematic  states.  This  is  essentially  a  Level  1 
function,  so  it  will  not  be  discussed  here.  Some  of  the 
definitions,  but  not  all,  include  the  requirement  of  know¬ 
ing  relationships  among  the  objects.  This  brings  three 
problems:  1)  The  relevance  problem:  there  are  so  many 
possible  relations — which  ones  are  relevant?  2)  The  re¬ 
source  problem:  where  can  we  get  the  necessary  infor¬ 
mation  resources,  both  data  and  processing,  that  can  be 
used  to  assess  the  current  situation?  3)  The  derivation 
problem:  how  do  we  derive  whether  a  particular  rela¬ 
tion  holds  or  not?  And  even  fewer  definitions  capture 
the  aspect  of  awareness  as  defined  in  the  Webster  dictio¬ 
nary,  where  awareness  is  explained  as  “AWARE  implies 
vigilance  in  observing  or  alertness  in  drawing  inferences 
from  what  one  experiences.”  In  other  words,  a  subject  is 
aware  if  the  subject  not  only  observes  (experiences)  the 
objects  but  also  is  capable  of  drawing  conclusions  from 
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Fig.  6.  SAWA.  Situation  management  component  (SMC),  relation 
monitor  agent  (RMA),  triples  data  base  (TDB),  and  event 
management  concept  (EMC). 

these  observations.  We  call  this  the  inference  problem: 
how  can  we  infer  the  implications  of  a  specific  situation 
on  the  tasks  that  we  are  pursuing? 

The  observations  presented  in  this  paper  have  been 
collected  during  the  two  year  period  of  working  on  the 
situation  awareness  assistant  (SAWA),  shown  in  Fig.  6 
[27],  In  most  general  terms,  SAWA  is  an  ontology  based 
situation  monitor  [26].  Its  main  goal  is  to  monitor  a 
“standing  relation,”  i.e.,  a  query  formulated  in  terms 
of  an  underlying  ontology.  SAWA  collects  informa¬ 
tion  (events)  and  invokes  its  inference  engine  that  de¬ 
rives  whether  the  relation  holds  or  not.  The  reasoning 
mechanism  of  SAWA  combines  logical  inference  with 
Bayesian  belief  propagation.  A  number  of  findings  from 
this  project  have  been  published  in  papers  [20,  21,  26, 
27,  28]. 

Solutions  are  sought  by  either  procedural  or  logical 
(declarative)  means.  In  the  logical  approach,  a  query 
about  a  specific  relation  can  be  posed  to  an  inference 
engine  (or  a  theorem  prover).  The  inference  engine 
then  returns  an  answer,  possibly  with  some  variable 
bindings.  A  number  of  inference  engines  for  OWL 
have  been  developed  and/or  are  under  development.  In 
typical  data  fusion  applications  the  derivation  problem 
is  solved  in  a  procedural  way,  i.e.,  in  order  to  determine 
whether  a  particular  relation  holds  or  not,  a  procedure  is 
invoked,  which  returns  either  a  “yes”  or  a  “no”  answer, 
possibly  also  including  some  return  parameters.  While 
this  approach  may  turn  out  to  be  more  efficient  in 
terms  of  time  complexity,  it  lacks  the  genericity  that 
the  logical  approach  has.  The  limitation  comes  from  the 
fact  that  only  those  queries  for  which  procedures  have 
been  explicitly  coded  by  the  system  developer  can  be 
answered.  The  logical  approach  is  termed  declarative 
programming,  while  the  procedural  approach  is  called 
procedural  programming. 

In  the  logical  approach,  the  inference  problem  is 
closely  related  to  the  relation  derivation  problem.  A  log¬ 
ical  query  regarding  any  feature  of  a  situation  is  posed 
to  an  inference  engine.  The  query  language  for  OWL  is 
called  OWL-QL.  The  number  of  types  of  queries  is  only 
limited  by  the  complexity  of  the  ontology  that  captures 
the  domain  knowledge.  The  queries  are  built  out  of  the 
class  expressions  and  property  expressions  using  logi¬ 


cal  connectives  that  are  part  of  the  ontology  language. 
Again,  the  advantage  of  the  logical  approach  is  that  the 
query  engine  is  not  designed  to  answer  a  specific  set 
of  queries,  but  it  is  rather  generic,  capable  of  answer¬ 
ing  any  query  that  is  expressible  in  the  query  language. 
This  is  not  the  case  in  the  procedural  approach,  where 
only  those  queries  that  have  been  formulated  at  the  de¬ 
sign  time  can  be  resolved  by  the  system.  The  reasoning 
mechanism  of  SAWA  combines  logical  inference  with 
Bayesian  belief  propagation.  Although  the  logical  ap¬ 
proach  is  a  promising  approach  to  solve  the  general  SA 
problem,  still,  a  number  of  issues  need  to  be  resolved 
in  order  to  make  the  logical  approach  scalable  up  to  the 
real  world  problems. 


Issues  for  SA 

Relations — Future  in  Semantic  Web  approach 

1 .  Relevance — need  a  generic  relevance  theory 

2.  Resource — from  closed  (level  1  provides  all  information)  to 
open  (level  2  accesses  Semantic  Web  for  additional 
knowledge) 

3.  Derivation/Inference — expressiveness  versus  efficiency  of 
reasoning 

Challenges  for  SA 

1.  Consistency  and  ontology  mapping 

2.  Identity  crisis  (association  problem) 

3.  Representational  expressiveness,  computational  complexity 

4.  Trust  Metrics 

5.  “Semantic  Web” — use  standard  language  (i.e.  "OWL”),  but 
need  more  expressiveness  (rules) 


3.  USERS  AND  APPLICATIONS 

3.1.  Tactical  SA  and  Computational  Support 

“Tactical  Situation  Assessment  Challenges  and 
Implications  for  Computational  Support”  Gerald  M. 
Powell 

A  number  of  definitions  of  Level  2  fusion  are  avail¬ 
able  [2,  3,  23].  The  comments  in  this  paper  relate  to 
one  or  more  of  these  definitions.  The  operational  fo¬ 
cus  is  Army  brigade  intelligence  analysis.  There  exist 
approaches  to  instantiate  these  definitions  into  practi¬ 
cal  designs  [3].  Fig.  7  shows  the  representational  in¬ 
formation  from  Waltz  [36]  which  shows  the  context- 
dependent  perceptual  knowledge  views  for  processing. 
These  displays  show  spatial  and  temporal  relations  from 
which  to  act.  The  display  technology  is  domain  depen¬ 
dent  and  requires  operational  considerations  [30-31]. 
What  follows  is  a  small  subset  of  key  issues  and  chal¬ 
lenges  in  knowledge  representation  and  reasoning  meth¬ 
ods  for  Level  2  fusion  in  this  task  domain. 

Hypotheses  and  Their  Utilities'.  There  is  a  need  to 
generate  hypotheses  to  serve  as  predictors  of  behav¬ 
ior,  to  guide  information  gathering,  and  to  provide  a 
framework  for  constructing  plausible  explanations  for 
evidence.  The  goal  of  creating  hypothesis  structures  that 
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DOMAIN  REPRESENTED 


VIEWS 


Representative  Examples 


COGNITIVE 

VIEWS 

Preventative 


SYMBOLIC 

VIEWS 

Predictive 


PHYSICAL 

VIEWS 

Reactive 


•Time  histories  of 
influence  nets  or 
agent-based 
simulation 
•  Locations  of 
threats 


•  Network 
information  flow 
simulation 

•  Comparison  of 
information 
transition  data 
flows 


•Spatial  Overlays 
•Current  FM-34- 
130  IPB  Spatial 
Overlay  Templates 
•  Physical  location 
of  objects 


Fig.  7.  Categories  of  view. 


will  satisfy  these  purposes  would  indicate  an  adequate 
understanding  exists  of  the  hypothesis  types  required 
and  the  logical  relationships  among  them.  The  relative 
importance  of  a  given  hypothesis  in  the  structure  will 
be  context-dependent — this  requires  analysis.  Similarly, 
the  relative  value  of  reports/data  from  differing  source 
types  and  instances  will  be  context-dependent.  Like  hy¬ 
potheses,  their  relevance  or  value,  will  vary  over  time 
as  the  situation  unfolds  including  what  information  has 
already  arrived,  whether  it  has  been  analyzed,  the  qual¬ 
ity  of  it,  its  relative  importance  and  so  on.  A  report 
deemed  irrelevant  in  a  particular  context  may,  much 
later,  become  relevant  as  the  interpretation  has  evolved. 
Analysis  and  interpretation  are  context-dependent.  They 
must  take  place  within  the  set  of  context-dependencies 
defined  by  the  information  peculiar  to  a  given  situation 
(mission,  terrain,  battlespace  reports,  etc.)  as  well  as 
historical  knowledge  about  the  adversary.  These  depen¬ 
dencies  can  cause  combinatoric  growth  in  the  number  of 
interpretations  possible  and  lead  to  erroneous  analyses 
and  conclusions.  Identifying  what  these  dependencies 
are,  and  constructing  ways  to  represent  and  reason  with 
them  to  produce  increased  accuracy  and  speed  of  anal¬ 
ysis  and  interpretation  remain  open  issues. 

Weak  Models  of  the  Adversary :  In  some  situations, 
our  knowledge  of  the  actors  we  are  observing  and 
trying  to  understand  may  be  extremely  weak  such  as 
when  there  has  been  little  opportunity  for  information 
gathering  prior  to  engagement,  when  their  organiza¬ 
tional  elements  and  communications  patterns  are  par¬ 
titioned  in  ways  that  inhibits  discovery  of  structure,  and 
when  their  doctrine  and  tactics  encourage  rapid,  adap¬ 
tive  changes  in  behavior  sometimes  manifesting  in  un¬ 
expected  ways.  Even  when  opportunities  for  observa¬ 
tion  are  plentiful,  accurately  interpreting  data  in  a  timely 
manner  may  be  extremely  difficult  due  to  indicators  that 
are  weak  discriminators  of  hypotheses.  These  issues  in¬ 
dicate  there  are  implications  for  both  directed  and  undi¬ 


rected  machine-based  knowledge  discovery.  Also,  mod¬ 
els  and  tools  to  help  analysts  understand  situational- 
specific  risks  of  Type  I  (false  positive)  and  II  (false- 
negative)  errors  in  interpretation  would  be  useful. 

Multiple  Inferencing  Strategies :  Abductive,  deduc¬ 
tive  and  inductive  inferencing  are  present  in  human 
performance  in  situation  assessment.  There  are  implica¬ 
tions  for  machine  capabilities  to  support  each  of  these 
in  an  integrated  framework.  Their  machine  implementa¬ 
tions  should  be  such  that  they  support  user  understand¬ 
ing,  trust  and  acceptance. 


Issues  for  SA 

1 .  Massive  information  overload  on  analysts 

2.  Analysis  and  interpretation  are  context-dependent 

3.  Cognitive  biases  cause  errors  in  analysis  and  interpretation 

4.  Models  of  adversary  structure/behavior  are  often  weak 

5.  Heterogeneous,  non-integrated  information  sources 

6.  Automated  environments  supporting  adequately  fast,  direct 
authoring  of  knowledge  by  analysts  do  not  exist 

Challenges  for  SA 

1 .  Automated  analysis/interpretation  that  is  fast  enough  while 
also  being  accurate 

2.  Overcoming  representational  and  processing  complexities 
caused  by  context-dependence 

3.  System  designs  that  will  help  analysts  overcome  cognitive 
biases 

4.  Automatic  adaptation  to  changing  threats 

5.  Semantic  consistency  across  info  sources 

6.  Building  adequate  knowledge  authoring  environments  for 
analysts 


3.2.  Urban  Combat 

“Situation  Assessment  in  Urban  Combat  Environ¬ 
ments”  Subrata  Das 

The  two  largest  hurdles  for  SA  in  contemporary  ur¬ 
ban  combat  environments  are  the  environmental  clut- 
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ter  and  the  enemy's  lack  of  conformity  to  established 
tactical  doctrine.  Adversarial  entities  in  the  environ¬ 
ment  must  be  identified  and  tracked,  individually  or  as 
groups,  to  recognize  higher-level  situations  (e.g.  attack, 
ambush,  interdiction,  insurgency)  and  determine  effec¬ 
tive  military  responses  or  preemptive  actions.  Further¬ 
more,  because  contemporary  enemy  behavior  is  often 
innovative  and  unpredictable,  traditional  tactical  mod¬ 
els  cannot  be  applied  to  recognize  significant  devel¬ 
opments  in  contemporary  situations.  As  a  result,  an 
effective  automated  means  for  extracting  useful  situ¬ 
ation  information  from  the  thousands  of  multi-source 
events  generated  every  minute  in  the  theatre  of  oper¬ 
ations  remains  an  open  problem.  Human  analysts  cur¬ 
rently  perform  the  bulk  of  this  difficult  situation  and 
threat  assessment  work,  but  are  only  able  to  process 
a  small  fraction  of  the  available  data.  Knowledge  dis¬ 
covery  (aka.  data  mining)  is  a  process  of  abstracting 
knowledge  from  data  to  form  models  for  problem  solv¬ 
ing.  Knowledge  discovery  techniques  such  as  Decision 
Trees  and  Inductive  Logic  Programming  discover  asso¬ 
ciation  rules  between  items  within  an  unordered  collec¬ 
tion  of  records,  transactions,  or  events.  Techniques  also 
exist  for  extracting  causal  Bayesian  belief  network  (BN) 
structures  along  with  their  strengths.  BN  technology  of¬ 
fers  several  advantages,  including  its  easy-to-understand 
graphical  modeling  and  consistent  probabilistic  seman¬ 
tics  in  dealing  with  the  uncertainty  involved  in  sensor 
data.  Focusing  now  on  the  model-based  approach,  cur¬ 
rent  state-of-the-art  approaches  for  answering  the  com¬ 
mander’s  priority  intelligence  requirements  (PIRs)  for 
SA  are  model-based.  Knowledge  discovery  or  model- 
based  approaches  fail  to  provide  a  complete  solution 
for  SA  requirements  because  a)  they  can  only  model 
specific  patterns  within  a  relatively  small  subset  of  vo¬ 
luminous  data,  and  b)  there  is  never  enough  historical 
data  available  to  model  novel  phenomena.  To  address 
these  issues,  we  explore  a  hybrid  approach  combin¬ 
ing  model-based  reasoning  with  knowledge  discovery 
techniques  for  SA,  especially  suitable  for  detecting  and 
identifying  asymmetric  threats  in  urban  environments. 
The  proposed  hybrid  approach  leverages  the  wealth  of 
data  available  to  provide  information  about  “what  is 
strange”  about  a  given  situation,  without  having  to  know 
what  exactly  what  it  is  we  are  looking  for,  thus  trigger¬ 
ing  models  for  follow-up  SA. 

The  hybrid  approach  recognizes  significant  patterns 
by  taking  into  account  environmental  clutter.  It  also  uses 
spatiotemporal  clustering  algorithms  to  perform  a  space 
and  time-series  analysis  of  messages  without  requiring 
semantic  information.  This  approach  can,  for  example, 
detect  spatially  correlated  moving  units  over  time  within 
the  environment.  Detected  patterns  trigger  follow-up  as¬ 
sessment  of  newly  developed  situations,  resulting  in  in¬ 
vocations  of  various  doctrine-based  computational  mod¬ 
els,  including  causal  static  and  dynamic  Bayesian  belief 
networks.  The  invoked  models  then  perform  SA  based 


on  other  observables  propagated  as  evidence  into  the 
models.  The  approach  extends  further  in  recognizing 
significant  patterns  without  relying  on  doctrinal  knowl¬ 
edge.  Instead,  we  make  use  of  latent  semantic  indexing 
(LSI),  which  is  a  proven  technique  in  text  based  infor¬ 
mation  retrieval  applications.  We  leverage  LSI  to  ex¬ 
tract  underlying  patterns  from  observables  reported  in 
formatted  (e.g.  USMTF)  or  plain  text  messages.  These 
patterns  establish  a  “normal”  profile  against  which  sub¬ 
sequent  incoming  observations  are  matched  so  as  to  de¬ 
tect  any  unusual  activities  (e.g.  large  scale  attack  prepa¬ 
ration). 


Issues  for  SA 

1.  Model  Based  Reasoning — closed  form  of  reasoning  and 
model  construction  process  is  time  consuming 

2.  Traditional  Knowledge  Discovery — requires  large  amount  of 
training  data 

3.  Link  Analysis — manual  process  and  not  able  to  handle  large 
amount  of  data 

Challenges  for  SA 

1.  Rapid  construction  of  models 

2.  BN — for  model  building  (all  pair-wise  interactions) 

3.  Unsupervised  clustering  techniques  for  large  volumes  of  data 
to  generate  normalcy  and  determine  “something  is  going  on” 

4.  Automation  to  find  "needle  in  a  haystack" 


3.3.  Collaborative  Situation  Awareness 

“Representation  and  Contribution-Integration  Chal¬ 
lenges  in  Collaborative  Situation  Assessment”  Daniel 
D.  Corkill 

Blackboard  systems  are  an  ideal  architecture  for  sit¬ 
uation  assessment  involving  large  data  volumes  and  het¬ 
erogeneous  data  and  knowledge  sources.  However,  the 
ad  hoc  confidence  and  belief  values  used  in  traditional 
blackboard  applications  have  led  to  criticism  of  the 
blackboard  approach  and  spawned  efforts  to  combine 
collaborative  blackboard- system  techniques  with  more 
“principled”  graphical-network  representations.  We  dis¬ 
cuss  two  important  collaborative-assessment  challenge 
areas:  1)  principled  blackboard  representations  and 
2)  principled  integration  of  contributions  made  by  inde¬ 
pendent  knowledge-source  entities.  The  complexity  of 
these  challenges  is  highlighted  using  a  simple  assess¬ 
ment  scenario,  shown  in  Fig.  8(a). 

The  effectiveness  of  blackboard  systems  is  the  prod¬ 
uct  of  a  number  of  architectural  capabilities  working 
in  concert.  The  first  important  capability  is  the  control 
flexibility  provided  by  indirect,  anonymous,  and  tem¬ 
porally  disjoint  interaction  among  software  entities.  The 
blackboard-system  control  shell  can  delay  execution  of 
a  knowledge- source  (KS)  execution  without  having  to 
modify  an  explicit  process  or  worry  about  managing 
the  data  needed  by  the  delayed  KS — they  remain  on 
the  blackboard.  Similarly,  KS  activations  can  be  exe- 
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cuted  earlier  than  normal — whenever  there  appears  to 
be  sufficient  information  for  them  to  perform  useful 
work.  Preliminary  efforts  in  applying  graphical  belief 
networks  to  blackboard  systems  have  focused  on  a  prin¬ 
cipled  representation  of  the  developing  solution  on  the 
blackboard  [34].  Current  beliefs  are  represented  on  the 
blackboard  as  disconnected  graphical  network  [9,  10, 
29],  The  emphasis  should  be  on  making  the  integra¬ 
tion  of  the  contributions  made  by  diverse  entities  well 
founded.  This  can  only  be  achieved  by  modeling  how 
these  contributions  are  generated  and  how  they  relate 
to  one  another.  For  example,  if  two  KSs  use  the  same 
data  and  produce  similar  results  using  different  compu¬ 
tational  approaches,  how  independent  are  the  results? 
Are  they  redundant  (with  no  added  certainty  in  the 
results)  or  complementary  (in  the  sense  that  each  has 
the  potential  to  make  mistakes  on  certain  data  values, 
but  these  mistakes  are  fully  independent  of  one  an¬ 
other)? 

The  Fair-Coin  Problem.  To  illustrate  these  chal¬ 
lenges,  consider  a  simple  collaborative-assessment  prob¬ 
lem  of  deciding  if  a  U.S.  quarter  is  a  fair  coin  (has  a  head 
and  a  tail)  by  observing  a  series  of  coin  flips.  A  priori 
we  are  told  that  there  is  a  50%  chance  that  the  quarter  is 
either  two-headed  or  two-tailed.  We  have  a  tabletop  that 
can  be  viewed  by  three  cameras:  two  black-and-white 
cameras  and  a  color  camera  (Fig.  8(a)).  Images  feed 
into  our  assessment  architecture  that  includes  a  number 
of  KSs.  There  are  low-level  KSs  that  attempt  to  identify 
coin  features,  higher-level  KSs  that  aggregate  features 
to  hypothesize  coin  sides,  and  a  decider  KS  that  makes 
the  fair  or  non-fair-coin  designation.  The  goal  is  to  make 
a  principled  determination  with  a  specific  confidence 
with  as  few  flip  observations  as  possible.  Adding  to  the 
complexity  is  the  U.S.  50  State  Quarter  program,  where 
a  new  quarter  with  a  state-specific  reverse  side  is  issued 
every  10  weeks  in  the  order  that  the  states  were  admitted 
into  the  Union  (Fig.  8(b)). 


Issues  for  SA 

1.  Blackboard  architectures 
Different  knowledge  sources 
Benefit  from  shared  information 
Bayesian  blackboard  systems 
Graphical  belief  nets  (procedural) 

2.  Integration  of  contribution  systems 

Challenges  for  SA 

1.  Representation  of  uncertainty  and  certainty 

2.  Develop  entity-specific  behavioral  specifications  of 
contributions 

3.  Specifications  provided  by  user  for  computer  to  learn 

4.  Development  of  feature-identification  Knowledge-source 

5.  Use  of  characteristics  in  concert 

6.  How  to  deal  with  mistakes  in  condition  characterizations 


3.4.  Human  Aided  Situation  Awareness 

“Human-Aided  Multi-Sensor  Fusion”  Enrique  H. 
Ruspini,  Artificial  Intelligence  Center,  SRI  Interna¬ 
tional 

In  multi-sensor  fusion  problems,  relevant  knowledge 
cannot  be  completely  represented  by  computer  models. 
In  these  cases,  it  is  necessary  to  implement  mechanisms 
that  permit  human  experts  to  apply  the  full  range  of 
knowledge  that  only  they  can  master.  We  identify  two 
fundamental  requirements  for  such  a  system.  It  is  first 
necessary  to  identify  properties  of  a  reasoning  system 
that  may  be  visualized  by  humans  so  as  to  judge  the 
credibility  and  reliability  of  its  results.  In  addition,  it  is 
necessary  to  implement  control  and  review  procedures 
that  may  be  applied  by  humans  to  improve  fusion 
results.  We  believe  that  any  sophisticated  human-aided 
multi-sensor  system  that  addresses  these  two  needs  must 
provide  the  following  capabilities: 

a)  Knowledge  acquisition  procedures 

b)  Explicit  representation  of  multi- sensor  knowledge 
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Applying  PAS/SEAS  to 
Missile  Defense  Discrimination 
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Fig.  9.  Structural  evidential  argumentation  system. 


c)  Quantitative  indicators  of  properties  of  fusion 
results 

d)  Intuitive,  understandable  displays  of  those  prop¬ 
erties 

e)  Interactive  techniques  to  improve  the  quality  of 
fusion  results 

We  propose  a  human-aided  multi-sensor  fusion  sys¬ 
tem  based  on  the  integration  of  the  Probabilistic  Argu¬ 
mentation  System  (PAS)  [4],  developed  by  Lockheed 
Martin,  and  the  Structural  Evidential  Argumentation 
System  (SEAS)  [25],  developed  by  SRI  International, 
shown  in  Fig.  9.  These  two  software  tools  implement 
variants  of  the  Dempster-Shafer  (DS)  calculus  of  evi¬ 
dence  [19].  PAS  is  a  formalism  that  explicitly  encodes 
assumptions  by  means  of  logical  rules  in  the  context  of 
a  generalized  probabilistic  framework.  The  reasoning 
procedures  of  PAS  produce  measures  of  support  and 
plausibility  for  various  conclusions  while  also  provid¬ 
ing  mechanisms  to  explain  the  nature  of  the  inferen¬ 
tial  chains  employed  to  arrive  at  those  results.  SEAS 
permits  the  recording  of  analytical  processes  employed 
by  intelligence  analysts  to  derive  their  findings.  SEAS 
was  originally  developed  to  support  collaborative  rea¬ 
soning  among  multiple  analysts.  SEAS  provides  intu¬ 
itive  graphical  displays  that  enable  analysts  to  review 
analytical  processes,  their  underlying  assumptions,  and 
the  nature  of  the  processes  employed  to  arrive  at  con¬ 
clusions.  In  practice,  the  structured-argumentation  pro¬ 
cesses  employed  by  SEAS  have  been  shown  to  facilitate 
quick  understanding  of  analytical  processes  while  per¬ 
mitting  capture  of  the  collective  thinking  of  groups  of 
analysts. 

The  integration  of  PAS  and  SEAS  attempts  to  satisfy 
the  previously  requirements  by  developing: 


a)  Logical  rules  to  facilitate  the  acquisition  and 
explicit  representation  of  knowledge 

b)  DS  calculus  of  evidence  to  provide  a  powerful 
mechanism  to  model  sensor  evidence  and  uncertain 
knowledge 

c)  Explanations  about  the  fusion  processes  to  per¬ 
mit  quantification  of  the  relevance  of  various  knowl¬ 
edge  items  and  the  detection  and  identification  of  con¬ 
tradictions  while  enabling  consideration  of  alternative 
hypotheses 

d)  Graphical  displays  to  facilitate  understanding  of 
inferential  chains  and  their  conclusions 

e)  Interactive  control  and  review  mechanisms  to 
permit  humans  to  correct  arguments  to  increase  the 
utility  of  conclusion  and  fusion  results 


Issues  for  SA 

1.  Knowledge  acquisition  systems 

2.  Explicit  representation  of  multi-sensor  knowledge 

3.  Quantitative  indicators  of  properties  of  results 

4.  Intuitive,  understandable  displays  of  those  properties 

5.  Interactive  techniques  to  improve  fusion  results  quality 

Challenges  for  SA 

1 .  Logical  rules  to  facilitate  acquisition 

2.  DS — evidence  for  uncertain  knowledge 

3.  Explanation  of  process — evidence  and  contradiction 

4.  Graphical  displays  to  facilitate  inferential  chains 

5.  Interactive  control  for  corrections  and  utility  of  conclusions 


3.5.  User  Refinement — Level  5  of  DFIG  Model 

“DFIG  Level  5  (User  Refinement)  issues  supporting 
Level  2  (Situation  Assessment).”  Erik  Blasch 
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Fig.  10.  DFIG  2004  model. 


The  current  fusion  model  supporting  the  evaluation 
and  deployment  of  sensor  fusion  systems  is  the  User- 
Fusion  model,  [7],  shown  in  Fig.  3,  with  upgrades  from 
the  current  Data  Fusion  Information  Group1  (DFIG) 
(which  is  the  current  JDL).  The  key  for  SA  is  the  user’s 
mental  model  [1].  The  mental  model  is  the  representa¬ 
tion  of  the  world  as  aggregated  through  the  data  gath¬ 
ering,  IF  design,  and  the  user’s  perception  of  the  social, 
political,  and  military  situations. 

The  DFIG  model,  shown  in  Fig.  10,  separates  the 
data  fusion  and  management  functions.  Management 
functions  are  divided  into  sensor  control,  platform 
placement,  and  user  selection  to  meet  mission  objec¬ 
tives.  Level  2  (SA)  includes  tacit  functions  which  are  in¬ 
ferred  from  level  1  explicit  representations  of  object  as¬ 
sessment.  Since  the  unobserved  aspects  of  the  SA  prob¬ 
lem  can  not  be  processed  by  a  computer,  user  knowl¬ 
edge  and  reasoning  is  necessary.  The  current  definitions, 
based  on  the  revised  JDL  fusion  model  [7],  include:  (see 
for  other  revisions  [24]) 

Level  0 — Data  Assessment :  estimation  and  predic¬ 
tion  of  signal/object  observable  states  on  the  basis  of 
pixel/signal  level  data  association  (e.g.  information  sys¬ 
tems  collections); 

Level  1 — Object  Assessment:  estimation  and  predic¬ 
tion  of  entity  states  on  the  basis  of  data  association, 
continuous  state  estimation  and  discrete  state  estimation 
(e.g.  data  processing); 

Level  2 — Situation  Assessment:  estimation  and  pre¬ 
diction  of  relations  among  entities,  to  include  force 
structure  and  force  relations,  communications,  etc.  (e.g. 
information  processing); 

Level  3 — Impact  Assessment:  estimation  and  pre¬ 
diction  of  effects  on  situations  of  planned  or  estimated 
actions  by  the  participants;  to  include  interactions  be¬ 
tween  action  plans  of  multiple  players  (e.g.  assessing 
threat  actions  to  planned  actions  and  mission  require¬ 
ments,  performance  evaluation); 


'Frank  White,  Otto  Kessler,  James  Llinas,  Alan  Steinberg,  Dave  Hall, 
Ed  Waltz,  Gerald  Powell.  Mike  Hinman,  John  Salerno,  Erik  Blasch, 
Dale  Walsh,  Chris  Bowman,  Mitch  Kokar,  Joe  Karalowski,  Richard 
Antony. 


Level  4 — Process  Refinement  (an  element  of  Re¬ 
source  Management):  adaptive  data  acquisition  and 
processing  to  support  sensing  objectives  (e.g.  sensor 
management  and  information  systems  dissemination, 
command/control) . 

Level  5 — User  Refinement  (an  element  of  Knowledge 
Management):  adaptive  determination  of  who  queries 
information  and  who  has  access  to  information  (e.g. 
information  operations)  and  adaptive  data  retrieved  and 
displayed  to  support  cognitive  decision  making  and 
actions  (e.g.  human  computer  interface). 

Level  6 — Mission  Management  (an  element  of  Plat¬ 
form  Management):  adaptive  determination  of  spatial- 
temporal  control  of  assets  (e.g.  airspace  operations)  and 
route  planning  and  goal  determination  to  support  team 
decision  making  and  actions  (e.g.  theater  operations) 
over  social,  economic,  and  political  constraints. 

For  SA,  the  user  must  (1)  prioritize  information 
needs  to  the  fusion  manager,  (2)  require  reliable  and 
validated  information,  and  (3)  seek  patterns  [6].  The 
information  priority  is  based  on  the  information  desired. 
The  user  must  have  the  ability  to  choose  or  select  the 
objects  of  interest  and  the  processes  from  which  the  raw 
data  is  converted  to  the  fused  data.  One  of  the  issues  in 
the  processing  of  fused  information  is  related  to  ability 
to  understand  the  information  origin  or  pedigree.  It  is 
important  to  note  that  reliability  and  validity  are  two 
different  concepts.  A  piece  of  information  can  be  100% 
reliable  and  either  totally  diagnostic  (100%  validity)  or 
un-diagnostic  (0%  validity)  in  predicting  information. 
However,  the  less  reliable  the  information,  the  less  valid 
it  is  because  of  the  inherent  uncertainty  (i.e.,  error)  in 
the  information  itself. 

Users  have  individual  differences  for  Reasoning 
Methods  (RM)  and  thus,  the  coordination  between  the 
user  and  the  machine  needs  to  be  flexible.  An  example 
is  that  one  user  might  look  at  sensor  data  while  another 
might  plan  missions  (see  Fig.  10).  The  responsibility 
of  the  user  thus  determines  the  information  needs  re¬ 
quirements  for  SA.  To  be  able  to  facilitate  many  users, 
a  control  strategy  needs  to  be  defined  wherein  the  user 
can  query  and  update  the  database.  One  way  to  facilitate 
user  opportunities,  a  standard  set  of  metrics  for  Knowl¬ 
edge  Representation  (KR)  should  be  designed  that  af¬ 
ford  Quality.  Blasch  [6]  explored  the  concepts  of  level 
2,  situation  awareness  or  assessment,  by  detailing  the 
user  needs  of  attention,  workload,  and  trust  which  can 
be  mapped  into  metrics  of  timeliness,  throughput,  con¬ 
fidence,  and  accuracy.  Table  II  lists  metrics  for  SAW  as 
referenced  to  the  communications,  human  factors,  auto¬ 
matic  target  recognition  (ATR),  and  target  tracking  lit¬ 
erature.  SA  is  hard  to  define  and  creates  interface  prob¬ 
lems  if  not  standardized.  Information  needs  of  fusion 
systems  for  KR  and  RM  need  rigorous  testing  in  ex¬ 
perimental  designs  to  define  SA  Products.  Additionally, 
dynamic  updating  of  Knowledge  Delivery  for  planning 
requires  timely  and  reliable  data  for  reasoning. 
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TABLE  II 

Metrics  for  Fusion  and  Situational  Awareness 


COMM 

Human  Factors 

Sit  Aware* 

ATR 

TRACK 

Delay 

Reaction  Time 

Timeliness 

Acquisition/Run  Time 

Update  Rate 

Probability  of  Error 

Confidence 

Confidence 

Prob.  (Hit),  Prob.  (FA) 

Probability  of  Detection 

Delay  Variation 

Attention 

Purity,  Precision 

Positional  Accuracy 

Covariance 

Throughput 

Workload 

Usage 

#  Images 

No.  Targets 

Cost 

Cost 

Utility 

Collection  platforms 

No.  Assets 

Security 

Trust 

Reliabilty 

Ontology,  Taxonomy 

Cooperative  Nav. 

*Tadda  et  al.  propose  some  of  these  for  Cyber  SA:  purity  for  quality  detection,  evidence  recall,  and  attack  score  [35]. 
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